Social networking application for real-time selection and sorting of photo and video content

ABSTRACT

Systems and methods of the present disclosure include a virtual wall having a plurality of media, as referred to as entries or media items. The system lays out the entries horizontally and vertically in the virtual wall. In the virtual wall, many carousels, or horizontal strips of pages, are stacked vertically, and gestures are detected to move vertically, from one horizontal strip to the next above or below. To navigate the display to the next carousel, the client device receives a user input to navigate between carousels, and transitions between the currently displayed media item to a media item in a different carousel. To navigate to the next media item within a carousel, the client device receives a user input to navigate between media items within a carousel, and transitions between the currently displayed media item and a next or previous media item within the carousel.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the priority benefit of U.S. ProvisionalApplication Ser. No. 62/382,699, filed on Sep. 1, 2016, entitled “SocialNetworking Application For Real-Time Selection And Sorting Of Photo AndVideo Content,” which is hereby incorporated by reference herein in itsentirety, including all references cited therein.

FIELD OF THE INVENTION

The present technology is directed to a social networking application,and more specifically, but not by limitation, to systems and methods forimplementing a virtual wall of media.

BACKGROUND

The approaches described in this section could be pursued but are notnecessarily approaches that have previously been conceived or pursued.Therefore, unless otherwise indicated, it should not be assumed that anyof the approaches described in this section qualify as prior art merelyby virtue of their inclusion in this section.

A common problem with computer applications is that there is not enoughroom to display all the media or information that user is interested in.For instance, the user may have a list of one hundred emails or onehundred TV shows they wish to browse through. The standard solution isto summarize these media entries into a list or grid of titles orsummaries, or small images, or even small animated images with titles orsummaries. However, previous solutions ignore an important aspect ofhuman behavior. Depending on the image, it may not be faster to processsmall images than large images. In addition, if the summary text isinadequate to actually summarize the material, it may not be easier orfaster to process short summaries versus longer or larger, but moreeasily understood previews.

For example, if a user is shopping for a dress, a small picture of adress may not provide sufficient information about that dress todetermine if the user is interested in that dress. A larger image oreven a video may more efficiently or effectively communicate. Whilelarger or longer media may solve one problem, it creates another withrespect to where to put the larger summaries, whether text, photo,video, or other media. Typical solutions have been some variant of thehyperlink. If a user is interested in something, the user clicks, tapsor otherwise selects a smaller version of that object, at which pointthe display changes to a larger version of that object. Such a solutionis not intuitive. Objects in physical space do not suddenly zoom orteleport in and out of view. Accordingly, there is a need for thedevelopment of systems and methods to effectively provide large amountsof information to a user with a natural and intuitive user interface.

SUMMARY

Embodiments of the present disclosure relate to systems and methods fortransforming and displaying media and messaging on a social network orother multi-user application. In various embodiments, the presentdisclosure includes systems and methods for efficiently displaying aplurality of media items in a virtual wall. The virtual wall maycomprise a plurality of rows or carousels, each row or carousel havingone or more related media items. The systems and methods includedisplaying a first media item of a first row or carousel on a userdevice, receiving a flicking gesture or other suitable gesture to theleft or right, and displaying a previous or next media item of thecurrently selected row or carousel. The systems and methods furtherinclude receiving a flicking gesture or other suitable gesture up ordown and displaying a default or determined default media item of theprevious or next row or carousel. In certain embodiments, the systemsand methods automatically display a determined next media item after apredetermined or calculated period of time.

In some embodiments, the systems and methods include generating anddisplaying one or more navigation user interfaces. A first navigationuser interface, otherwise referred to as a vertical navigator,facilitates a user quickly vertically scrolling through and selecting amedia sequence or row from a plurality of sequences or carousels. Asecond navigation user interface, otherwise referred to as a horizontalnavigator, facilitates a user quickly horizontally scrolling through andselecting a media item of the one or more media items from within acarousel.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, where like reference numerals refer toidentical or functionally similar elements throughout the separateviews, together with the detailed description below, are incorporated inand form part of the specification, and serve to further illustrateembodiments of concepts that include the claimed disclosure, and explainvarious principles and advantages of those embodiments.

The methods and systems disclosed herein have been represented whereappropriate by conventional symbols in the drawings, showing only thosespecific details that are pertinent to understanding the embodiments ofthe present disclosure so as not to obscure the disclosure with detailsthat will be readily apparent to those of ordinary skill in the arthaving the benefit of the description herein.

FIG. 1 illustrates a diagrammatic representation of a virtual wall,according to the present disclosure.

FIG. 2 shows a high-level block diagram of an exemplary social messagingnetwork 200, according to the present disclosure.

FIGS. 3-6 depict exemplary graphical user interfaces displaying a mediaitem of the virtual wall illustrated in FIG. 1, according to the presentdisclosure.

FIG. 7 illustrates the graphical user interface having a prompt with aslider for adjusting an audio volume setting of the client device,according to the present disclosure.

FIG. 8 shows another exemplary graphical user interface, according tothe present disclosure.

FIG. 9 depicts a representation of a summary of a virtual wall,according to the present disclosure.

FIG. 10 illustrates various states of a carousel state indicator button,according to the present disclosure.

FIG. 11 shows a graphical user interface that facilitates mediarecording and sharing, according to the present disclosure.

FIG. 12 depicts an exemplary graphical user interface that facilitatestrimming videos, according to the present disclosure.

FIG. 13 illustrates a further exemplary graphical user interface havinga vertical navigation user interface, according to the presentdisclosure.

FIG. 14 illustrates a further exemplary graphical user interface havinga horizontal navigation user interface, according to the presentdisclosure.

FIG. 15 shows a high-level block diagram of an exemplary video streamingsystem, otherwise referred to as a virtual stadium, according to thepresent disclosure.

FIG. 16 illustrates a graphical user interface 1600 for implementingreaction to reaction chat overlays, according to the present disclosure.

FIG. 17 illustrates an exemplary computer system that may be used toimplement some or all embodiments of the system.

DETAILED DESCRIPTION

The following detailed description includes references to theaccompanying drawings, which form a part of the detailed description.The drawings show illustrations in accordance with exemplaryembodiments. These example embodiments, which are also referred toherein as “examples,” are described in enough detail to enable thoseskilled in the art to practice the present subject matter. Theembodiments can be combined, other embodiments can be utilized, orstructural, logical, and electrical changes can be made withoutdeparting from the scope of what is claimed. The following detaileddescription is, therefore, not to be taken in a limiting sense, and thescope is defined by the appended claims and their equivalents.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one. In this document, the term“or” is used to refer to a nonexclusive “or,” such that “A or B”includes “A but not B,” “B but not A,” and “A and B,” unless otherwiseindicated. Furthermore, all publications, patents, and patent documentsreferred to in this document are incorporated by reference herein intheir entirety, as though individually incorporated by reference. In theevent of inconsistent usages between this document and those documentsso incorporated by reference, the usage in the incorporated reference(s)should be considered supplementary to that of this document; forirreconcilable inconsistencies, the usage in this document controls.

Various embodiments described herein relate to systems and methods forproviding real-time selection and sorting of media content.

In some embodiments, systems and methods of the present disclosureinclude a virtual wall having a plurality of media, as referred to asentries or media items. The system, a client device for example, laysout the entries, typically at full size, horizontally and vertically inthe virtual wall. The user may navigate through the entries as rapidlyas they wish as tiles or pages in the virtual wall. This is animprovement on an older, one-dimensional solution called a carousel. Inthe carousel, a page mechanism allows users to see pages, which maycontain images or even videos laid out from left to right in aconsistent order, by ‘going’ left or right. A similar, existingone-dimensional solution is to read entries by navigating up and down,otherwise known as a vertical strip.

The virtual wall may display the largest possible images, videos orpages, and in a preferred embodiment, displays them at full screen. Noadditional zoom/un-zoom step is necessary to display media at its fullsize, nor is any scrolling required, as the tiles or entries in thevirtual wall are already properly scaled, to fit perfectly within thevirtual wall. Unlike the more primitive one dimensional solutions, orcomplex mixed systems of scrolling, zooming and navigation, the virtualwall facilitates near instant access in two dimensions, effectivelylaying out pages or media in a two dimensional virtual wall that doesnot require complex zoom or scroll commands. The User taps or flicks orotherwise indicates their desire to navigate to the next item above,below, right or left. Instead of hiding objects ‘too big’ or ‘too many’to fit onto the screen into a hyperspace joined with hyperlinks, thevirtual wall makes it extremely easy and fast to navigate from videoscene to scene, image to image, or page to page, and from video tovideo, album to album, or chapter to chapter.

FIG. 1 illustrates a diagrammatic representation of a virtual wall 100.The virtual wall 100 is a two dimensional, arrangement of media (videos,photos, text or combinations of the above, etc.) in full-sized (oftenfull screen) tiles or pages (also referred to as media items).

In certain applications, pages or tiles are often arranged into strips,otherwise known as media sequences or carousels (such as carousels 105a-f), from left to right, on which simple interfaces or gestures areused to move through the pages from horizontally, from left to right orright to left. Each carousel 105 a-f may represent an event, show,chapter, article, episode, story, etc.

In the virtual wall 100, many carousels 105 a-f, or horizontal strips ofpages, are stacked vertically, and simple gestures are used to movevertically, from one horizontal strip to the next above or below.Up/down buttons or gestures move up and down between carousels, unliketraditional interfaces, where up/down gestures or interfaces scroll apage up and down. In the present embodiment, to navigate the display 115to the next carousel, for example from a second carousel 105 b to athird carousel 105 c, a swipe down gesture or go down button can bepressed by the user and determined by a processor of a client device. Inthe present example, media item 110 a is currently displayed asrepresented by display 115. The client device receives a user input tonavigate between carousels, and in response to the user inputtransitions between the currently displayed media item 110 a to a mediaitem in a different carousel, such as media items 110 d-e. To navigateto the next media item within a carousel (such as to the next page, clipor sequence within a multiple page/part chapter, article, episode,story, etc.), a swipe right gesture or go right button can be pressed bythe user and determined by the client device. For example, the clientdevice receives a user input to navigate between media items within acarousel, and in response to the user input transitions between thecurrently displayed media item 110 a and a next or previous media itemwithin the carousel, such as media items 110 b-c.

Unlike the standard hyperlink arrangement in which media is not mappedto a two-dimensional space, in the virtual wall 100 each media elementis located in a specific row and column. The arrangement in the virtualwall 100 is much easier to navigate and recall than the traditionalhyperlink system, where objects do not have a clear location in a twodimensional space. However, unlike a simple 2D grid of tiles, each rowworks as a carousel, so while up-down navigation increments ordecrements the vertical coordinate of which tile or page being view isdisplayed, the horizontal coordinate depends on the position of thatrow's carousel. As shown in FIG. 1, the vertical axis is represented ascolumn 120, within which the display 115 is moved up and down from onecarousel to the next, otherwise described as moving from one verticalcoordinate to the next. Diagrammatically, each carousel 105 a-f shiftseither left or right to display each media item of the currently viewedcarousel, or to shift the horizontal coordinates of the currently viewedcarousel. It is to be understood that each carousel may have a varyingnumber of media items.

In order to simplify navigating the virtual wall 100, in one or moreembodiment, navigation may be limited to either vertical or horizontalmovement, but not both at once. Thus, for example, if the userswipes/mouses/gestures mostly left but also up, the up component isignored. The client device may determine a directional vector of theuser gesture, determine the largest component of the directional vectorin a horizontal or vertical direction, and further select the directionof the largest component to navigate the display 115 to another mediaitem either in the same or different carousel. It is to be understoodthat, while one or more embodiments throughout the present disclosurerefer to directions such as left, right, up and down, the virtual wall100 may be flipped or rotated in any suitable orientation. For example,media items within a carousel may be oriented between a first and asecond opposite direction, such as left and right respectively, along afirst directional axis, while media items between different carousels(such as along column 120) may be oriented in a third and fourthopposite direction, such as up and down respectively, along a seconddirectional axis.

The virtual wall 100 includes several other important advantages overthe standard hyperlink solution. Since, unlike Hyperlinks, there are alimited number of possible objects to directly navigate to (in mostembodiments there are four objects—up/down/left/right, but more arepossible with additional navigation elements), the client device canmore easily provide a ‘look ahead’ media caches, particularly to thenext page or tile, to reduce any latency or lag view images/views. Inother words, the client device stores in memory the media itemscorresponding to the next and previous media items in a currentcarousel, and the media items to be displayed for a next and previouscarousel. Such media items are referred to as being adjacent media itemsto the currently displayed media item.

The virtual wall is not just a two dimensional grid of large tiles, withtwo important differences. The first difference is that in a grid, allrows are the same length. In the virtual wall 100, rows are as long asnecessary to contain the elements on each row. The second difference ishow row-to-row navigation works. In a grid, navigation up or down doesnot change the column, only the row. In the virtual wall 100, navigationup or down, in some embodiments, also changes the column. In someembodiments, moving down a row also moves the column back to theleftmost position. Other, more sophisticated embodiments are describedbelow.

The virtual wall 100 facilitates remembering ‘where’ an object is in thevirtual wall 100. With hyperlink objects in hyperspace—which has noparticular correspondence to real space, it is very difficult orimpossible for a user to utilize the user's natural ability to rememberwhere things are placed—ahead, behind, left, right, after, before,above, below, near, etc. On the contrary, by using the virtual walltechnique, entries have a place. So as the user transverses through thevirtual wall 100, it is simple for both the user and the client deviceto remember and store in memory, for each row, which column the userlast viewed, and when the user returns to that row, display that column.In other words, the client device stores in memory a column position ofthe last displayed media item for a carousel, for each carousel, whichhelps act as a reinforcing mnemonic element for the user. Furthermore,since media objects are laid out in a 2D space, the virtual wall 100facilitates creating smaller one or two-dimensional ‘summaries’ of partsof the larger ‘virtual wall’, as will be described in greater detailbelow.

The virtual wall 100 may implement various methods of organizing mediacontent or other items. In some embodiments, the virtual wall 100displays items in order of relevance, importance or quality. Items withhigher scores, or move votes, etc., will be sorted from left or right,or up or down, or nearer or further from the current position in thevirtual wall 100. In some embodiments, the virtual wall 100 providesmedia content by time or inherent order. Media items that followsequentially will be ordered in that sequence. That is, a first item ina row may correspond to the beginning of a row, a story, a video, a setof choices, etc. The next item, for example to the right, corresponds tothe next choice, in space or in time. Moving further in the samedirection, for example to the right, moves through the list, from startto end. Moves left, or in the opposite direction, will move up or backthrough the list. In some embodiments, the start and end or end andstart are connected like a circular carousel.

In some embodiments, a first displayed media item of a row or carousel,otherwise referred to as a starting element, is not necessarily theleftmost media item of the list. In one embodiment, the first displayedelement in a row is the one the user most recently viewed, unless thatelement was viewed to completion, in which case the next element in thatrow is displayed. The next element after the final, rightmost element,may be the first, or leftmost element, effectively ‘wrapping around’that row. The client device may determine the first displayed media itembased on the user and the situation. For example, the starting element,or all elements, can be randomly or algorithmically selected from thelist of items. Randomizing the elements facilitates that elementsreceive approximately the same attention. In a further embodiment, theclient device may determine the first displayed item is the item havingthe greatest ranking or rating of the items. For example, the clientdevice may highlight a particular element, for instance the top choiceout of twenty choices, by selecting the top choice as the startingposition. In some embodiments, to draw attention to a new element, or‘page’ with new information, the client device places the new item inthe leftmost position, or the client device sets the carousel positionso that the new item is the first item to be displayed.

In certain embodiments, the client device may deemphasize one or moremedia items of a row or carousel. For example, certain media items mayhave already been seen by the user, have low scoring, have beenidentified as not useful or hidden, or otherwise indicated to bedeemphasized. In some embodiments, the system may determine not todisplay any deemphasized items, or to skip over them, or displaying fora short amount of time, or flowing over them more quickly than normal.

While navigation from carousel to carousel may go from a start positionin a first carousel to a start position in a second carousel, this maynot be ideal in all situations. In certain embodiments, the clientdevice stores in memory the last position of each carousel. However, inone or more embodiments, if the media in the last position hascompletely played, the first carousel associated with that media isadvanced to the next position. If the user navigates off the firstcarousel, as the client device has completely played the media, the lastposition of the first carousel is modified to the next element.

In some embodiments, the system determines a predetermined ordering ofentries within a carousel (typically left-right) and from carousel tocarousel (typically up-down). The inter-carousel arrangement may bebased on time or sequence (that is, the most recent item is at the top)and older items are below, though the reverse is also possible. Theclient device may either manually or automatically advance, from left toright, top to bottom or in some other simple progression, as otherwisedescribed in the present disclosure.

FIG. 2 illustrates a high-level block diagram of an exemplary socialmessaging network 200 having a social messaging service or system 210,at least one sender client device 220 (e.g. first user), at least onerecipient client device 230 (e.g. second user, third user, to Nth user)and a communications network 240. In general, the client devices 220,230 may relate to a wide range of electronic devices including, but notlimited to, a smartphone, computer, desktop computer, laptop computer,tablet computer, PDA, wireless telephone, cellular phone, televisionsystem, remote controller, gaming console, gaming pad, in-vehiclecomputer, infotainment system, smart home computer, smart watch,wearable computing device, and any other suitable electronic devicehaving at least one processing or computational unit. In variousembodiments, the client device may be equipped with a digital videocamera configured to capture digital media such as photo and/or videoand to transmit the digital media to a remote location. In addition, theclient device may have various input/output devices for interactionswith a user, including but not limited to a touch screen display,keyboard, key pad, trackball, touchpad, display, monitor, television,projection system, augmented or virtual reality system, or othersuitable input/output device. The client devices 220, 230 may beimplemented by exemplary computer systems such as those shown anddescribed in FIG. 17. The exemplary social messaging network 200 isfurther described in related United States Patent Application “SocialMessaging System For Real-Time Selection And Sorting Of Photo And VideoContent” (application Ser. No. 14/998,106) filed Dec. 23, 2015, which ishereby incorporated by reference for all purposes. It is to beunderstood that the client devices 220, 230 may implement one or more ofthe methods of the present disclosure, including generating and storingthe virtual wall 100. While many embodiments describe the client deviceperforming certain steps, it is to be understood that one of ordinaryskill in the art would understand some or all of the recited steps maybe performed on the social messaging service or system 210.

FIGS. 3-6 depict an exemplary graphical user interface 300 displaying amedia item from the virtual wall 100. The displayed media item, firstmedia item 305 a, may be the media item 110 a as represented in FIG. 1.

FIG. 4 illustrates an exemplary transition of the graphical userinterface 300 from the first media item 305 a to a second media item 305b within a carousel. The transition may be the transition from mediaitem 110 a to media item 110 c within carousel 105 b as shown anddescribed in the diagrammatic representation of FIG. 1, for example. Theadvancement from the first media item 305 a to the second media item 305b may be automatic, manual, or a combination of both.

In one or more embodiments, a finger flick gesture from right to leftadvances the current view to the next entry 305 b in the carousel, and afinger flick gesture from left to right changes the current view back tothe previous entry 305 a. FIG. 4 depicts a frame of a transition betweendisplaying the first media item 305 a and the second media item 305 b inresponse to a finger flick gesture from right to left on the graphicaluser interface 300 of FIG. 3. It is to be understood that a finger flickgesture may be one of many suitable user inputs, such as pressing anarrow button, received and determined by a processor of the clientdevice to trigger the transition between media items and carousels.

FIG. 5 shows a further example of the graphical user interface 300, inwhich a finger flick gesture from bottom to top starts a next carouseland a finger flick gesture from top to bottom starts the previouscarousel. In particular, in response to receiving the finger flickgesture, or other suitable user input, the client device transitions thedisplay of the first media item 305 a of a first carousel (as shown inFIG. 3) to a first media item 305 c of a second carousel (as shown inFIG. 6). Other information associated with the media item or carouselmay also transition from the first media item 305 a to the first mediaitem 305 c. For example, a first title 310 a or description of the firstcarousel will update, via the client device and graphical user interface300, to a second title 310 b of the second carousel. Furthermore, chatinformation in a chat overlay 315 may be associated with a particularmedia item, and will update upon display of a new media item.

The media items may be arranged both horizontally and vertically so thatthe user's swipe, flick, or other direct manipulation correspondsdirectly to a two dimensional location of the media item they wish tonavigate to. In some embodiments, in response to the user gesturing tothe next media item when the currently displayed media item is a lastmedia item of the current carousel, the current view will appear toshift normally (shift from right to left) but will display a first mediaitem of the subsequent carousel. As such, the graphical user interface300 may display each media item of the carousel sequentially and willloop to the next carousel upon reaching the last media item.

In various embodiments, the virtual wall 100 is used to display videos.Unlike static images or page layouts, videos have a time component orduration. When videos are completed, in traditional implementations,they stop, pause or loop. All of these options break the continuity ofuser experience. Alternatively, upon completion of playback of a video,the tile displayed on the virtual wall 100 slides off to the left and anew tile, representing the next video to be displayed, slides in fromthe left. The next video may be a new video or a next part of a longervideo broken into separate discrete pieces. The new video may start toplay the moment the previous video finishes playback, so that the userreceives a continuous video experience, which is often highly desirable.

In some embodiments, if the last video or media item in a carousel playsto the end, or is manually advanced, instead of pausing, stopping, orwrapping around to the first media item in that carousel, the displaymay automatically transition to the next item in the next carousel, thatis vertically, in a continuous manner. The transition may be similar totransitioning from one video to the next video horizontally within acarousel, or in other embodiments, the completed video or media item mayslide left and up as the next video or media item slides in from theright. The inverse action may take place if the user goes backwards.Similarly, if the user wishes to skip ahead, or skip back to the next orprevious row in the wall, the user may navigate up or down. In bothcases, the transition from one carousel to the next carousel enables anearly seamless selection and play of related content in an intuitivemanner, by arranging them vertically adjacent.

In some embodiments, the client device determines a duration each mediaitem is displayed on the graphical user interface 300 beforeautomatically proceeding to the next media item of a carousel. In thesimplest case, the duration may be the length of the video, or a fixedamount for viewing a photo. However additional time may be necessary, toeither read or listen to comments added to the media item or to giveusers time to make a decision to vote, or other interface interactions.

Users may need some time to make a decision about a media item. If themedia item is of short duration, and if the media item automaticallyadvances after play, the user may not have time to make a decisionbefore the media item is advanced to the next media item. In someembodiments, the client device loops the media item until there issufficient time to review and interact with the media item as desired.For example, in the case where at least 10 seconds is needed to view andreact, and a video is only 4 seconds long, the client device may loopthe video multiple times. Each time the media item completes, the clientdevice may determine if a total time the media item has been displayedis less than the predetermined duration for the user to view and react.If not enough time has passed, the client device loops the video, orstays on the image. Otherwise, if the total time the media item has beendisplayed is greater than the predetermined duration, the methodsadvance to the next media item.

FIG. 6 further illustrates that, in addition, a media item in a carouselmay include additional layered information; such as a chat overlay 315.The additional layered information enables the user to combine media andmessaging seamlessly.

There are several well know methods of reacting to, commenting on, orotherwise providing feedback to media. For recorded media, typicallycomments are created and appear below or in a different area than themedia. With such methods there is no attempt to match the comments tothe part or scene of the media the comments might correspond to. Forlive media, typically comments are created as the media is being watchedand are time stamped so that the comments can be synchronized to themedia at the associated time stamp. This method, unfortunately, put aburden on the creation creator to exactly time his or her reaction. Thetypical result is that comments are created late, often after the timein the media that the reaction is appropriate too. Additional commentsfurther complicate the situation as it becomes difficult to create orreview multiple comments quickly.

In various embodiments, the client device uses video capture to createaudio or video reactions, referred to herein as reactions (for examplereaction 325 displayed in the chat overlay 315). Unlike standard audioor video messaging, reactions are designed to provide commentary ordiscussion about the associated underlying media, or as a reply to aprevious message, not as standalone messages.

Unlike standard messaging, reactions are created over the relevantmedia. Creating a reaction while currently viewing the associated mediaitem is a far simpler process than creating a reaction, and thensubsequently finding the right media and right time to attach thereaction to. In some embodiments, in response to determining a userinput such as a gesture, command or other UI element indicative of adesire to create a reaction, the reaction UI is displayed over the mediawhich the user wishes to react. In some embodiments, the client devicepauses the underlying video in response to generating and displaying thereaction UI.

In some embodiments users may be able to suppress the playback of thereactions though a gesture or interface, such as a distraction-freeinterface.

If the reaction has audio, and a volume of the client device is off ortoo low to be heard, or set in a caption mode, in some embodiments, theclient device may use speech to text methods to display the text of theaudio. To help ensure that audio reactions can be heard, the clientdevice may reduce the volume of the underlying video. If the underlyingvideo ends before the reaction is played to its end, the client devicemay loop the underlying video. The system may automatically playreactions, starting from a first video reaction (which may be the one atthe top) and play in time sequence from there to preserve the order ofthe conversation. Alternatively, if there are too many reactions to playall of them, the system may skip older or less important reactions.

In response to receiving a user input associated with pausing, in apreferred embodiments, the client device pauses both the main video andthe reaction videos, to ensure that the user has a simple and fast wayof freezing and silencing their application

Also, in some embodiments, if the user taps a small reaction image orvideo, or otherwise indicates interest in a particular reaction, theclient device visually expands the reaction significantly, in oneembodiment, as large as will fit on the screen. Furthermore, in responseto the user expanding the reaction to full screen, the client devicepauses the underlying video, thus reducing its volume to zero. In oneembodiment, tapping on a large reaction will return it to normal size.In related embodiments, if a large reaction has been shrunk, it is alsopaused, and the next reaction, if any, is played.

In one embodiment, if there are multiple reaction videos to play, onceone reaction video is enlarged, the user can advance to the nextreaction, or go back to the previous reaction, with appropriate gesturesor buttons, such as a swipe gesture.

In another embodiment, reactions play one at a time. While the reactionsmay play in order, from top to bottom, left to right, etc., the user cantap to skip any reaction to skip to that specific reaction.

In certain embodiments, in response to receiving a scrolling interfaceor gesture, the client device scrolls the chat window 320 that overlaysthe media item 305 a. If the scrolling the chat window 320 bringsadditional reactions on screen, the client device may automatically playthe additional reactions. The client device stores a record of whichreactions have and have not played so that previously played reactionscan be skipped.

In other embodiments, navigating through a list of reactions can be donewith an interface described below, the vertical navigator.

In various embodiments of the present disclosure, long videos aresegmented by the client device into a series of shorter, scenes orclips, generally with some easy way to play and navigate through theseclips, such as the virtual wall 100 in FIG. 1. Once longer media hasbeen split up into multiple parts, it is far easier to provide feedbackor reactions on each segment or clip. If the user chooses to interactwith a particular clip, the client device may automatically pause orloop the clip at the beginning of the interaction process, giving theuser as much time as desired to complete their interaction. If the userchooses to review reactions, those reactions can be displayed over therelated media segment, so that the user will easily recognize theparticular segment or clip the reaction is associated with. If thevirtual wall 100 is used, instead of a single screen to contain allreactions to an event, chapter, show, etc. , the client device displaysone screen per segment or shot, not only tying the reactions to aspecific shot or scene, but perhaps more importantly, also offering moreroom for each reaction (e.g. one area per scene, segment or page, versusone area for the entire event). In addition, users providing feedbackare not forced to race to react or consume reactions while the video isadvancing.

The client device may generate a chat room, or other environmentsuitable for commentary and reactions, such as the chat overlay 315. Invarious embodiments, the client device transforms each clip or shortvideo into a background video with sound, as a background for the chatroom. Each media item acts as a chat room or a messaging area for allratings, votes, polls, comments, reactions, information or purchaserequests, etc. In some embodiments, the systems and methods of thepresent disclosure provide a chat room associated with each media item.The systems and methods may receive text comments, images, audio orvideo reactions to the media item and display the received reactionsover or near the media item. The systems and methods may be coupled toone or more social graphs.

In some embodiments, if a user begins to enter a reaction about oneclip, or is reviewing reactions about one clip, the client device doesnot advance to the next clip until the user has completed creating orconsuming the reactions, and, instead either pause the underlying media,or in one or more embodiments, loop the underlying media during thecreation and review of reactions pertinent to that clip. Once the clipis complete, and all desired reactions are reviewed, the client deviceautomatically advances to the next clip.

A disadvantage of displaying reactions over media is that the clutter ornoise may be annoying. In some embodiments, the client device determinesactuation of a button or gesture, such as a long tap or force tap, andin response hides or suppresses most or all the overlay items (forexample chat overlay 315), and actuation of another button or gesture,such as any tap, re-displays the overlay interfaces. In conjunction withthe virtual wall, in one embodiment, once the overlay interface ishidden, the overlay interface remains hidden until the sequence or ‘row’of the virtual wall has completely played, or until the overlayinterface brought back manually. In another embodiment, the overlayinterface remains hidden across multiple virtual wall strips, untilbrought back manually.

Various embodiments enable playing reaction audio or video. In oneembodiment, the client device auto-plays all un-played, on-screenreactions (audio or video), from the top, down, unless interrupted byuser action or other events. After auto-playing the reactions over eachentry, the client device resumes auto-move. In another embodiment, theusers determines if and when they want to trigger play of reactions viaa button, gesture or voice command. In one embodiment, as each reactionis played, the reaction scrolls up, and if the reaction scrolls up andout of the reaction window, the reaction disappears. In someembodiments, the client device may provide an option for each user toadjust their reading speed.

In some embodiments, reaction videos or audio play at a normal orpredetermined pace. In some embodiments reaction videos or audios playback faster, either by a system default or a user preference. In someembodiments, users can select specific reactions to play or replay bytapping on them, or scrolling to them and then tapping.

In some embodiments, the system stores data associated with whichreactions have been played, and sets the scroll position of the reactionwindow so that the highest un-played or un-seen reaction is displayed.In response to the user leaving an entry or media item, and laterreturning to the same entry or media item, the client device startsauto-playing the reactions on the next un-played or un-viewed reaction.

If all reactions have been played, and a new reaction is received by theclient device associated with the currently viewed media item, in apreferred embodiment, the client device plays the new reaction as soonas possible, or immediately after completion of playback of thecurrently viewed reaction. In response to receiving a plurality of newreactions, the client device plays the plurality of new reactions insequential order. If new reactions are available to play, the clientdevice may skip down to the new reactions.

In one embodiment, once a user creates a reaction, the created reactionis displayed in the chat overlay 315, or user's comment stream, andimmediately played. In some embodiments, the reaction is created on theclient device, and displayed by the client device via the graphical userinterface upon creation, whether or not the created reaction has beentransmitted to the server.

In a preferred embodiment, a user selects which reaction to play bytapping on the reaction in the chat overlay 315. In response, the clientdevice plays the reaction, and/or if necessary, stopping the playback ofsome other reaction.

While the time to review audio or video reactions can be equated to thelength in time of the media, a particular consideration is how muchreview time is needed for text comments or images. In one embodiment,the review time is based on an approximated length of time to consumethe reaction. If this time estimate is too high, and the viewer does notwhich to wait, in response to a simple flick, or other gesture orindication, the client device will advance to the next entry or scene.

The user may take longer to see, read or digest the additional oroverlay information than the time to see the image or watch theunderlying video of the media item. If the additional information isalready timed, such as an audio or video clip, the time to play thisinformation can be considered. If the information is static, the clientdevice may determine a number of words, letters or other informationsymbols, or lines, and from this determine a display duration such thatthe user has enough time to consume the information. In certainembodiments, the display duration is the amount of time a media item isdisplayed before advancing to the next media item. For example, in thecase of English text, the client device may determine the duration todisplay the media item and extend the duration for addition informationover that media by example, for three seconds per line to be read, or atenth of a second per letter to be read. In the example, if there areone hundred letters in additional information displayed over the mediaitem, then the methods display the media for at least one hundredmultiplied by a tenth of a second, or ten seconds.

In order to further simplify the display, not all layered informationmay be displayed or played at the same time. For instance, we may wantto restrict the display of layered information to a certain window 320of the graphical user interface 300. As sufficient time passes toconsume that information, the informational layer can scroll up, so thatadditional information can be seen or heard. A related problem is, ifthere is more layered information than can fit on the screen, where tostart the display of this information. One approach is to start at thetop, and scroll down as sufficient time to consume the layeredinformation has passed, or to give the user manual control overscrolling layered information. In a preferred embodiment, the user isgiven manual control, and the client device advances the layeredinformation at a rate which the user can adequately consume theinformation, and if there is more information than easily consumed, theclient device prioritizes which layered information is displayed, anddisplays the most recent information, or most important, as appropriate.

In some embodiments the client device may adjust the duration of a mediaitem such that the user can digest additional information on a firstview, but on repeat viewings allow less time to review already presentedinformation. The system determines which information displayed on amedia item is new additional information or previously presentedinformation. In response to a user viewing information displayed on anmedia item, the system records in a database whether the information hasalready been viewed or not. New information (e.g. has not beenpresented) is given a full amount of time to digest per unit ofinformation. Information that has already been presented is given lesstime to digest. In certain embodiments, the amount of time may be thelength of the audio or video commentary, or a number lines or words perseconds. For example, new comments may be allocated one second per wordto read or review, but previously seen comments may only be allowed onesecond per three words. If the display of information is scrolled, thistiming information can be used to control the scroll rate.

A concern of automatically advancing media is that sometimes users needto interact with the media item or the media application, for instanceto add a comment, rate the media, vote, like or delete the media, suchthat the media remains on the current view while the user is attemptingto interact with the media. One solution is to pause the media duringany user interaction. However, pausing will stop the media playback,which is not always desirable. In various embodiments, the client deviceextends the time before advancing to the media item by looping the mediainstead of pausing the media. In the preferred embodiment, if the partof the media is still visible the client device continues to play themedia, looping if necessary, but if the media is entirely covered byother aspects of the application, the client device pauses the media,and restarts play once the media again visible or partially visible. Theclient device detects user interaction. In response to detecting userinteraction, the client device extends the display duration of the mediaitem, and continues to extend until the user interaction is completed.For example, if a user brings up a menu that does not entirely cover themedia, or triggers a vote animation, the carousel does not advance tothe next media item until the menu is closed or the vote animation iscomplete.

A further concern of automatic advancement through potential videochoices is that users need a reasonable amount of time to make adecision. For instance, a quiz or game show may desire to give the usereight seconds to make a decision about which selection to choose. Insome embodiments, the client device determines if a total duration todisplay every media item in a carousel is less than a predeterminedduration indicative of the amount of time a user needs to make adecision. The client device may determine the total duration by summingthe display duration of each media item of a carousel, which may includeshort audio clips, videos or photos with short delays. If the totalduration is less than the predetermined duration to make a decision,then the client device repeats the media sequence until thepredetermined duration is reached.

In various embodiments, if the predetermined duration is reached but themedia has not reached an end of playback, then the media repeats playinguntil the end is reached. If the user wishes to accelerate the process,she is able, using the virtual wall mechanisms described herein (orother, common mechanisms) to skip ahead.

Unlike traditional approaches which are designed to navigate and displaypassive media, approaches of the present disclosure are designed tonavigate and display active media, media with additional commentary orfeedback, which is more complex and requires new solutions. The systemof the present disclosure facilitates viewing both passive and activemedia. The user may interact with the content, as long as desired,without preventing the content from advancing when the user is passive.The client device comprises a video time (VT) and a minimum time (MT).The video time represents a length of time of a media item, if a video,or the minimum time, that is a predetermined period of time, such as afew seconds, if the media item is an image or other static media.

Furthermore, certain actions may delay the next auto-move. The timer forauto-move is referred to as an auto-timer, and the time to hold on anentry is referred to as an auto-delay.

Some actions, such as voting or closing the on-screen keyboard (OSK) ormenu, trigger a reasonable minimum delay or grace period beforeauto-moving. These actions reset the auto-delay=Max (current auto-delay,MT).

Auto-delay is the length of time the system remains on a media itembefore auto-moving. The minimum auto-delay of videos or audio is theduration of the video or audio. For photos or rich pages, the minimumauto-delay is set appropriately by the client device based on how muchtime the creator wishes to give, by default, to the viewer. In someembodiments the minimum auto-delay is a fixed amount for all photos orby line of text. Additional time may be added to consume comments orother reactions to the media. If the video finishes playing before theauto-delay is complete, the client device may either repeat the video orpause. In certain embodiments, in response to a gesture or interfaceelement, the client device prevents auto-moves, such gestures mayinclude holding a fingertip on the image or video, or other suitablegestures.

Unread comments or unseen reactions may extend auto-delay for sufficienttime to read or process the unread comments or reactions. The clientdevice may determine the sufficient length of time based on the time,size, rows, length or type of unread comments or reactions. If there aremore unread comments or reactions than can be displayed on one screen,the client device may exclude them from determined length of time toextend the auto-delay.

Various actions may interrupt/extend the auto-timer. The client devicepauses the auto-timer (but does not turn off auto-move indicator) duringthe interruptions. In response to an end of the interruption, the clientdevice restarts the auto-timer. Some interruptions reset the auto-timer.In resets, if the auto-time remaining is less than MT, the client deviceresets the auto-time to the MT. This is referred to as resetting theauto-timer.

FIG. 7 illustrates the graphical user interface 300 having a prompt 705with a slider 710 for adjusting an audio volume setting of the clientdevice. The client device may generate and display the prompt uponopening the application, or returning to the application from adifferent application. In some embodiments, an audio volume setting of amedia item is reduced by a predetermined amount if the media item isplayed repeatedly. While audio may greatly add to media experiences,sometimes the audio can annoy the user. For example, when short medialoops, the sound track often becomes annoying on the second, third orlater loops. The client device may automatically reduce the audio volumesetting of a media item if the media item is looped or repeated. In someembodiments, the audio volume setting of the media item is dropped fiftypercent each time the media item is looped.

FIG. 8 shows a media item 805 as a landscape video. A common issue ofapplications that play media is that the media may not be formatted toperfectly fit the form factor of the replay device. In variousembodiments, the client device displays portrait media in portrait mode,even if the display is in landscape mode, and landscape media inlandscape mode, even if the display is in portrait mode. As shown inFIG. 8, while the device is oriented in portrait mode, and all of theinterface elements on the graphical user interface 300 are oriented forportrait mode, the media item 805 is shown in landscape mode. In otherwords, the user interface may be displayed normally, while media isrotated sideways. In some embodiments, the client device rotates themedia if the rotated orientation enables a significantly larger image(for instance 25% more pixels). This non-intuitive rotation helpspreserve the solidity of the virtual wall display.

In various embodiments of the virtual wall 100, the client device maydisplay images and videos that are not shaped to match the shape of thescreen with padding as necessary to avoid clipping the media. In certainembodiments, however, the client device may zoom up non-full screenimages and videos to better preserve the solidity of the virtual walldisplay.

FIG. 9 depicts a representation of a summary 900 of a virtual wall, suchas virtual wall 100. An advantage of the virtual wall 100 is that thevirtual wall 100 is easily summarized, by shrinking the full screentiles or pages down to a fraction of their normal size, and trimmingthem, but retaining the same two dimensional layout of the full sizedtiles or pages. For example, each carousel 905 may be arrangedvertically, while media items 910 of each carousel 905 are arrangedhorizontally.

Another advantage of the virtual wall 100 is that it is easy to extendcarousels by add additional pages or tiles to any horizontal strip. Forexample, the client device may determine an actuation of an add gesture,command or button, such as button 915, and trigger a prompt to the userto either select media stored in the client device or activate a camerato capture new media. The additional tile representative of the newmedia item may be positioned at the last or rightmost position, orimmediately after the current tile, if that is desired. The buttons 915represent placeholders for adding these additional media elements. Inparticular, if the media items in the carousel represent a conversation,adding new material or media to the rightmost position can intuitivelycontinue the conversation. Furthermore, the summary 900 may include abutton 920 for creating a new carousel 905.

The virtual wall 100 also makes it easy to remove individual mediaelements by removing or hiding specific tiles or pages, either from thesummary 900 or the graphical user interface 300. Undesirable or removedtiles can be removed entirely, or quickly skipped over duringtransitions from one media item to the next, or blurred or covered up,as desired. A simple button (e.g. trashcan) or gesture (e.g. two fingerpinch) can be detected to remove or hide unwanted media items.

FIG. 10 depicts various states of a carousel state indicator button 810,as shown in FIG. 8. In certain embodiments, how a media consumer hasresponded to a media item is stored in memory and is displayed near orover the relevant media item as a carousel state indicator button 810.In particular, a method may include prompting a media consumer torespond to a media item. Upon determining that the media consumer hasresponded, the method may display what response the media consumerprovided.

Referring back to FIG. 1, the virtual wall 100 in certain embodimentsfacilitates generating catalog navigation or shopping. In pre-existingmethods, an online merchant user might present to the shopper one ormore items of interest, and a selection of navigational elements tochoose, for example, shoes, hats or coats. The user would click onvarious hotspots, some of which would hyperlink to additional pages. Ina physical store, however, the user would walk in and see items laid outin groups, perhaps shoes in one area, hats in another, coats in a third.If the user was interested in shoes, the user would navigate to thatsection, scan through the items in some physical order, perhaps fromleft to right, or top down, etc., pick it up and examine it, and/orcarry it to the checkout counter.

The virtual wall 100 can be used to offer a more intuitive catalognavigation or shopping experience. Each item in the catalog has at leastone corresponding tile or page in the virtual wall 100. Each tile orpage may comprise text, images or videos, or other suitable media. Eachdirectory or group of items in the catalog has at least onecorresponding tile or page in the virtual wall 100. Each directory mapsto a row or carousel of the virtual wall 100. Large directories, in someembodiments, may map to multiple rows. The virtual wall 100 may containany number of catalogs. Each catalog may consist of many directories andmany items. To navigate through a catalog, in the preferred embodiment,navigating up, via an up gesture or other up interface elementcorresponds to navigation up one directory. Similarly, navigating leftor right corresponds to navigation back and forth, through a directory.In some embodiments, large directories may also be navigated by up ordown interfaces or gestures. If the currently displayed page or tilerepresents a group or directory, navigating down corresponds to goinginto that directory. If the currently displayed page represents an item,navigating down, in some embodiments, may correspond navigating toadditional information about that item, information that may or may notbe contained within the virtual wall 100. In the preferred embodiment,if the additional information is not contained in the virtual wall 100,a user interface element or gesture enables returning to the same spoton the virtual wall 100 with a flick up, or back gesture or button.

In some embodiments, navigating a virtual catalog may correspond tonavigating a file system directory. In one embodiment, the top row ofeach catalog in the user's virtual wall 100 maps to the highest-leveldirectory of the catalog. The highest level may represent a series ofshops, departments, collections or other categories. The user, via thegraphical user interface of the client device, may navigate throughhighest level of choices by flicking left/right/up/down through theseries of large or full screen tiles, pages, images or videos, or usingthe other navigation interfaces described herein, or using a traditionalnavigation mechanism. In response to receiving a particular user input,such as a flick down or other suitable gesture or selection mechanism,the client device may select or ‘dive into’ a particular categoryassociated with the displayed page or tile. In some embodiments,flicking up from the highest level exits the catalog, and returns theuser to the carousel above the catalog. Navigating down, with thedescribed virtual wall navigation elements, from a directory orcollection displays the row or carousel that corresponds to the filesand subdirectories or sub-collections of that directory or collection.Navigating down, from the bottom of a directory or file, in someembodiments, will display additional information on the item associatedwith the displayed page or tile, or work similar to a hyperlink,redirecting the client device to another page with additionalinformation on the corresponding item.

If the user selects a subdirectory or collection in the first row, thevirtual wall tile will transition and display information on theselected directory, collection or item, such as shoes, coats, and hats.The user may quickly flick through the second row of entries, and ifinterested in a particular entry such as shoes, flick down on the entry.For example, the entry associated with the shoes directory may depict avideo, perhaps a video of someone spending a few seconds walking throughthe actual shoe department. For instance, a carousel associated with theshoe department may comprise three videos (or pages/tiles), one forsandals, one for boots and one for high heels. The user may quicklyflick or automatically advance through the photos or videos in thestrip, for example through to high heels. After diving into the heelsentry, the client device may present the user with various brands ofheels with another horizontal strip or carousel of full screen pages orvideos. The user may quickly flick or automatically advance to the brandof interest, and again flick down to dive in further. The client devicemay then present a horizontal strip of models of that brand. Again, theuser may select a particular model by flicking down. The system may thenpresent a horizontal strip of choices related to that item, for instancecolors. In response to the user flicking down or otherwise selecting acolor, thus completing the selection process, the system may present achoice of checking out (the screen might again slide down) or continueshopping (flick in any other direction). To exit the store, the usercould flick up until they get to the ‘mall’ level, or tap the leavebutton, or use other traditional methods to exit, such as selectinghome. If the user exited the virtual ‘store’ with items in their‘shopping cart’ upon exit from the store, the user could be reminded to‘check out’ via various suitable interfaces.

In various embodiments, the virtual wall 100 is used to display acatalog of pages, images or videos, which may correspond to product orservice offerings. In one embodiment, the catalog is mapped to adirectory structure. In this mode, traversing left or right through thevirtual wall 100 maps to traversing through a particular directory. Theorder of the items in the catalog may be set, or sorted by a value, likeimportance or alphabetical. Traversing up the virtual wall 100 maps togoing back up through the directory structure. Traversing down thevirtual wall 100 maps to going down through the directory structure.Directories may include files, links or additional directories. If thevirtual wall item corresponds to a directory or link, then a previewvideo or image that corresponds to that directory or link is displayedin the virtual wall 100. If the virtual wall item corresponds to a file,that file is displayed, with additional metadata as appropriate.

If the user attempts to navigate down when viewing a file, which is the‘bottom’ of a directory tree, various embodiments are possible. Thesimplest is stopping at the ‘bottom’ with some appropriate userfeedback. In a preferred embodiment, in response to receiving a userinput associated with navigating down further when viewing the bottomitem of a directory tree, the client device displays additionalinformation or actions regarding that item (which may include aninterface to save or purchase the item or request additional informationregarding the item), or navigating to a tile which asks the user whataction he or she desires, or navigating to the next virtual wall itemdown past the current directory structure.

FIG. 11 depicts a graphical user interface 1100 that facilitates mediarecording and sharing. A problem with capturing, editing or respondingto video in real time is that humans need approximately 150 ms-200 ms toreact to stimulus. If the user wants to create, split, trim an existingvideo, or restart video capture in real time, then a manual triggeraction of the user may be delayed by one or many video frames from anintended initial video frame. In various embodiments, in response to auser starting a video, or marking a video to be split or trimmed, themethods record the manual marked time and subtract a predeterminedreaction time. For example, a predetermined reaction time may be 150 ms.Any time mark used to start, split or trim video, the methods mayincorporate the predetermined reaction time into account, so that thestart/split/trim is based not on the manual touch time, but that touchtime less the predetermined reaction time.

In various embodiments, the client device facilitates splitting longvideos into short shots or segments, suitable for interaction includingcommentary. In one embodiment, after a reasonable amount of recordingtime for one shot, perhaps 15-60 seconds, the client device may splitthe recording into an additional recording, or provide standard tools tosplits longer shots into multiple shorter segments, or use othertechniques to automatically generate at which points to split longervideos into shorter videos, such as silence detection. In someembodiments, in response to splitting longer shots, the client devicemay add one or more visual or sound effects, as each shot is firstsplit.

Referring back to FIG. 11, the graphical user interface 1100 may includea send count indicator 1110 that represents a number of media consumersthat the contest will be sent to and/or visible by. The send countindicator 1110 clearly shows the number of media consumers that themedia creator is sending the contest to. In some embodiments, the sendcount indicator 1110 is represented as a badge over a portion of a sendbutton.

FIG. 12 depicts an exemplary graphical user interface 1200 thatfacilitates trimming videos. A single video trigger can capture timeboth before and after the trigger. For example, a video trigger cancapture five seconds before the video trigger. Then, in someembodiments, the client device presents to the user, via the graphicaluser interface 1200, with an option to further refine the start or addadditional chapter or split markets on the video with a video editbutton. If the user chooses to edit the clip, the client device presentsthe user with the longer acquired video, with the trim points selectedby default, and then gives the choice to move the trim points forward orbackward, or add additional chapter or split marks.

In various embodiments, the present disclosure is directed to fastintra-video and inter-video navigation systems and methods. Thetraditional solutions are to navigate through programs, channels orvideo lists using up and down buttons, by entering channel names ornumbers, or entering the name of the video, finding matching results andthen selecting one, if there are multiple matches. Unfortunately, theseare all poorly adapted for touchscreen devices. Accordingly, there is aneed for the development of a system and method for navigation thatprovides ease of use across devices.

FIGS. 13-14 shows a graphical user interface 1300 having a main window1305 through which a series of media is viewed. In various embodiments,the media, comprising images, video, etc., may be arranged horizontallyor vertically, as described in other various embodiments of the presentdisclosure. Each item of media includes a position along a first axis.The graphical user interface 1300 may further comprise one or morenavigation user interfaces, such as vertical navigation user interface1310 and horizontal navigation user interface 1315, each navigation userinterface having a plurality of previews 1320. The system couples aposition of the main window with a position of the plurality ofpreviews. In response to the user scrolling through the plurality ofpreviews 1320, the position of the main window 1305 moves to match theposition of the plurality of previews 1320. The client device couplingthe main window 1305 to the currently selected preview of the pluralityof previews 1320 facilitates speed and accuracy of media selection. Forexample, media item 1325 may correspond to the preview labeled “D,” andmedia item 1330 may correspond to the preview labeled “E.” The mainwindow 1305 may contain either playing or stopped videos. The user maysimply navigate through the plurality of previews 1320 and release whenthe desired choice is selected. It is to be understood that the user mayscroll through the plurality of previews 1320 through one or moregestures on a touch display or a virtual reality or augmented realitydisplay.

In some embodiments, the currently selected preview, while the user isnavigating the plurality of previews, is highlighted (for example,preview “D” in FIG. 13). In response to the user releasing their fingerfrom the touch display or otherwise finalizing the selection, the clientdevice removes the navigation user interface 1310 and plays the selectedvideo or continues to display the selected image. While the navigationuser interface 1310 is displayed, the client device may darken orotherwise de-accentuate whole or parts of the underlying video (forexample, the system may use a mask that is darker under the list thanaway from the list) to make it easier to see what choices the user has.In some embodiments, the system plays the audio track of the primary orsecondary video, such as media item 1325 and 1330 respectively, or somecombination, the selection of which may be a default setting or a userdefined setting.

In some embodiments, the system provides annotations to one or more ofthe previews 1335 to clarify which media have already been played, beenpreselected by or for the user, are free or require payments, etc. Asshown in FIG. 13, the annotations may include a dot to indicate new,heart to indicate favorites, etc.

FIG. 14 illustrates a similar navigation user interface 1315 fornavigating within a video or sequence of related videos. Existingapproaches use a fast forward or rewind button, or a timeline controlused to move a video forward or backwards in time. The existingapproaches are not suitable for fast and flexible navigation of mediacontent on across devices.

In various embodiments, system includes a vertical navigation userinterface 1310 or inter-video navigator (as shown in FIG. 13), and ahorizontal navigation user interface 1315, or intra-video navigationuser interface or intra-video navigator (as shown in FIG. 14). Thevertical navigation user interface 1310 facilitates the user selecting aparticular sequence of media or carousel from a plurality of sequencesor carousels. The user or program may trigger or otherwise evoke thevertical navigation display by using a standard interface like a tap orswipe on an area of the graphical user interface 1300 that holds thevertical navigation user interface 1310. The horizontal navigation userinterface 1315 is displayed on the graphical user interface 1300, thehorizontal navigation user interface 1315 having a series of previews(or images) 1320 that correspond to a representative frame for thatalternative spot or alternative video, or otherwise media items of thecurrent carousel. It is to be understood that, while letters are shownas distinguishing various media items and previews for ease ofillustration, the media items and previews would depict the underlyingmedia, either as images, video, etc. In response to receiving scrolling,swiping or other suitable gesture or user input, the client devicescrolls through the series of images which facilitates the user movingthrough the series of choices quickly.

Similar to the vertical navigation user interface 1310, the horizontalnavigation user interface 1315 also couples the position of the mainwindow 1305 with a position of the series of previews 1320. In responseto either releasing a finger or pointer, tapping on a specific choice,or other suitable gesture or user input, the client device plays theselected choice. Similar to the vertical navigation user interface 1310,the horizontal navigation user interface 1315 facilitates viewing orwatching the video and simultaneously selecting a new position in thatvideo or other video.

In various embodiments, the previews quickly disappear once theselection is released, but not so quickly that there is not time for theuser to pick up their finger or selector, and continue scrolling,dragging or selecting. For example, in response to a selection, theclient device may wait a predetermined amount of time before hiding thenavigation user interface 1310, 1315. This facilitates ‘infinite’scrolling of choices, seamlessly.

Referring back to FIG. 1, embodiments of the present disclosure also maybe combined with live audio or video, such as live video or live gamebroadcasts, referred to herein as a live event. In some embodiments, thevirtual wall 100 is applied to the live event. Using the virtual wall100, the current live shot is the rightmost tile or entry in thecarousel or row that corresponds to that live event.

In some embodiments, the virtual wall 100 facilitates rewinding or‘going back in time’ in the live event. If the user navigates to a liveevent, and goes to the right-most or last entry, the client device willpresent to the user a live feed. Otherwise, if the user navigates to anyother entry for the live event, such as previous entries within acarousel associated with the live event, the client device willpresented to the user one or more recording of the event in the past.

Users often will want to ensure they are watching the live component ofa live event. The user may navigate to the rightmost entry to find thelive or latest component as previously described. In some embodiments,in response to a user input indicative of navigating right past thelatest or live entry on the right, the client device, instead ofnavigating to the next entry in the virtual wall, may display optionsrelated to the live stream, and/or synchronize the display with the livestream. However, user input indicative of navigating down to the nextrow of the virtual wall will navigate to that item.

In some embodiments, in order to turn a long event into a series ofshorter shots or videos, suitable for a carousel or virtual walldisplay, the client device or server will either manually orautomatically separate or split up a long video sequence into a seriesof shorter shorts. In some embodiments the client device generates abutton or gesture, such as a shake gesture, to split the recording intoanother entry at the point currently displayed. Furthermore, the clientdevice may utilize a timer to ensure that no single shot is too long.For instance, in response to a video shot reaching sixty seconds orother suitable predetermined length of time, the first sixty seconds isautomatically split into two separate segments or shots. In someembodiments, a graphical user interface is generated and displayed bythe client device to review sequences of shots and add, remove or changewhere the video transitions from shot to shot. In some embodiments, inorder to highlight the live shot, a word or symbol is displayed by thegraphical user interface over the entry associated with the live orpresent video.

In some embodiments, if the user pauses while watching a live event, thevideo is paused, and thus the user falls behind real-time. In apreferred embodiment, in response to the user un-pausing, the clientdevice may play the stream where the user left off in a new media entry,and also generate a new media entry in the virtual wall to the right,associated with the live event. The user may then navigate or advanceforward in time to catch up or join the live stream.

In some embodiments, instead of queuing up reactions, when watching livematerial it may be preferable to play reactions to that material live,as the client device receives the reactions. In some embodiments, theclient device limits the number of simultaneous reaction, for instance,up to three simultaneous reactions. Additional simultaneous reactions,may be queued in the comment flow. Reactions may be prioritized, and lowpriority reactions may be skipped.

Furthermore, reactions to live audio or video is more complex, asdelaying reactions may render them, if not pointless, at least out ofsynchronization from what the reactor intended. In typical systems,reactions received live during live broadcasts play as soon as possible.When too many reactions are received simultaneously the additionalreactions are typically queued up and played later when there is time orspace. However this technique has obvious limits as humans can only reador listen at limited speeds.

If a user is viewing a live-stream, and the live stream ends, but otherrelated live-streams are presently streaming, the client device mayadvance a position in the virtual wall 100 to the next relatedlive-stream. For example, if there are multiple live-streams of the sameevent, if one live-stream for that event ends, the virtual wall isadvanced to the next live-stream of the same event.

In further embodiments, the live event may correspond to live videofeeds from multiple screen captures or cameras, either associated withthe client device or remote devices transmitting video feeds via anetwork and/or server. Referring to the virtual wall 100, each carouselmay be associated with a particular camera, in which the last orright-most entry is a live feed, and previous entries are associatedwith previous segments of recorded video from that camera. For example,move/swipe/button left and right, or other suitable gestures or userinput, are forward or back in time, and up and down user input switchesthe current display between cameras or screen captures recorded withinthe same event, and during the same time period as the recording aboveor below. The client device responds to such user input as previouslydescribed herein. Furthermore, a vertical navigator as shown in FIG. 13may comprise small picture in picture images/videos as previews todisplay the live feeds received from multiple cameras.

FIG. 15 illustrates a high-level block diagram of an exemplary videostreaming system, referred to herein as a virtual stadium, having avideo or screen capture source 1510, mixer server or distribution point1520, a plurality of terminal or client devices 1530, and acommunications network 1540. The video source 1510 may be a live eventor broadcast which is being transmitted to the mixer server 1520 overthe communications network 1540, or alternatively may be a pre-recordedvideo either transmitted to the mixer server 1520 or hosted at the mixerserver 1520. There may be separate video servers for the primary event,and as well as audio mixing servers for the rooms and crowd. Forinstance, audio servers could be closer to the crowd than the servertransmitting the video source. Screen capture or video capture isavailable.

Currently, a popular form of virtual audience participation in massmedia is the text message or some form of “like” button. Typically thosetext messages or “likes” may be displayed over or near the media.

Embodiments of the present disclosure take fundamentally differentapproaches to audience participation. Instead of only presenting userfeedback as text or symbols, the virtual stadium also harnesses audio.Using audio, users may see a performance without text messages or othersymbols covering or distracting from the performance, while hearing andfeeling the presence of the audience.

For each virtual stadium 1500, the system involves a multiplicity ofvirtual participants, each virtual participant (or audience) associatedwith client computer hardware, such as a mobile phone, headset, computeror media player, some of which are equipped with one or moremicrophones, connected to a network. The client device associated witheach virtual participant may be referred to as terminals 1530,collectively referred to as the audience. Using techniques known tothose skilled in the art, participants receive the media, whethercreated almost live or previously recorded, at approximately the sametime, such as TV broadcasting, internet broadcast, or other mass mediadistribution systems. Zero, one or many participants may have activemicrophones recording their audio reactions to the mass media as theparticipants experience the media.

In some embodiments, the audience may be presented, via the terminals1530, with buttons or gestures to activate other audio reactions, suchas applause, booing, cheering, etc. For instance double tap may triggerclapping, a hard tap a cheer, a shake—booing, a drawn heart—a kisssound, etc.

Microphones may be turned activated or turned off by various well knownmechanisms, either automatically or manually, by such standard methodsas push to talk, turning on or off a microphone button, voice or volumedetection, etc. The microphone(s) capture live audio, typically theuser's voice and other sounds picked up by the microphone(s). The clientdevice(s) may process the audio input to improve a signal quality ortransmission efficiency of the audio input, using the methods of echoremoval, volume balancing, compression, etc. known to those skilled inthe art.

The audio signal may be time-stamped with the time of origination tofacilitate processing later in the mixer server 1520. After audioprocessing on an originating client, the originating client sends theaudio input, via the communications network 1540, to a server orcollection of servers referred to as mixer server 1520. To facilitateanalysis or mixing, the audio may be sent uncompressed via theoriginating client. To facilitate transmission, the audio may be sentcompressed via the originating client. In some embodiments, the audiosignal can be sent both compressed and uncompressed to optimizeprocessing speed over bandwidth. The audio input may be sent as a file,stream or other standard methods.

The mixer server 1520 receives a multiplicity of audio inputs for thevirtual stadium from the client devices 1530. The mixer server 1520mixes together the audio inputs, also referred to as streams, asappropriate, and sends one or more streams back out to all clients whoare listening to the virtual audience via client devices 1530. Feedbackservers may be separate from broadcast servers. In response to receivingthe streams, the client device 1530 plays both the mass media, andvirtual audience participation audio, which may be in separate streams,or mixed together into a single stream, in approximate synchronizationto create the effect of an audience providing nearly real-time audiofeedback to the media. If gestures or buttons are used to create othersounds, the other sounds, in a preferred embodiment, are synthesizedusing standard methods, such as playing a digital recording of thesound. In such embodiments, triggers for generating the synthesizedsounds are received by the originating client device and sent from theoriginating client device though the communications network 1540 to themixer server 1520, which mixes the synthesized sounds into the virtualaudience participation audio. To minimize overhead, the mixer server1520 may accumulate or sum the synthesized sounds into counters, and thecounts of the synthesized sounds may be sent instead of the synthesizedsounds themselves.

Complex issues may arise if, for example, hundreds or audience membersare making noise, and thousands are listening. In one embodiment, themixer server 1520 may create a single ‘crowd’ mix. The mixer server 1520combines all audio input equally and transmits the combined audio inputto all clients who have indicated their desire to play the crowd sound.

Unfortunately, many voices make it difficult to hear any particularvoice, such as the voice of a close friend. To mitigate this problem, insome embodiments, audio from a limited number of selected users, or‘voices’, are sent directly, peer to peer, or indirectly through theserver(s), using standard methods, in a separate mix or channel, to allparticipants, or just selected participants, so that the volume of theseselected voices can be controlled separately, and if necessary, thevolume of selected voices boosted, or the volume of the crowd reduced or‘ducked’ so that the selected voices can be understood. If too manyvoices are sent simultaneously to be understood, then the number ofselected streams may be throttled back to a reasonable maximum, say lessthan four voices. In some embodiments, the audio for a group of friendscould be given a separate channel or mix, to facilitate these friends totalk to each other, over the noise of the crowd.

In some embodiments the mixer server 1520 counts a number of voices bydetermining a number of received audio input streams from the terminals1530 and prioritizes the individually received audio input streams basedon various criteria, such as how close the virtual speakers are to thevirtual listener, either in location in the virtual stadium space, indistance in the listener's social graph, etc. In some embodiments, audiofrom close friends could be prioritized over audio from acquaintances.

In some embodiments, virtual participants automatically or manuallyselect a virtual location to speak or listen from via each terminal1530. In some embodiments, the mixer server 1520 assigns eachparticipants a virtual ‘seat’, a location in a virtual two orthree-dimensional virtual space.

In some embodiments, the mixer server 1520 assigns a virtual participantto one or more virtual rooms. Virtual participants in the same virtualroom would hear each other at a normal volume. Users in other virtualrooms would be played at a reduced volume. The mixer server 1520 maycombine audio inputs from terminals 1530 of virtual participants in thesame virtual room into a single virtual room audio stream, adjust avolume of the audio stream, and transmit the audio stream to theterminals 1530 associated with the virtual room. In certain embodiments,the volume, stereo mix, head related transforms, room parameters orother standard spatial audio techniques can be used to create the effectof a virtual room or space.

Depending on the number of participants, it may be difficult to create aunique audio mix for every participant. In order to increase efficiency,the audio of participants who are not near in the virtual space can bemixed together into a single stream or several streams. The audio ofnearby virtual participants, or participants who are loud enough to beunderstood, which may be defined as a predetermined virtual distance orvirtual room, are separately mixed, using standard mixing techniques. Inthe preferred embodiment, nearby or important virtual participants willbe louder than more remote or less important virtual participants.Finally, the nearby and not near mixes are mixed together for a finalmix and played on the client device. It is to be understood that themixing can take place in any device with sufficient computational power.

For a user, it is useful to see the person who is talking, for instance,in the event that the user wants to respond directly to that person, ormute that person. In some embodiments, when specific voices aretransmitted at sufficient volume to understand them, the name or avatarof the speaker may be displayed. This visual representation can then beused as a button to trigger additional information or interfacesrelating to that speaker, to review or replay their audio, to have aprivate conversation with that speaker, to change that speaker's volumeor mute the speaker, to block the speaker, etc.

One complication of audio mixes is that people do not like to hearthemselves delayed by more than a few hundred milliseconds. In room orlocation based mixes, if everyone in the location or group receives thesame mix, then people may hear themselves, perhaps significantlydelayed. In some embodiments, when a client transmits audio, the exacttiming of that client's audio is recorded by the client and transmittedwith the audio. Before playback, on either the client or the mixerserver 1520, the client's own audio stream is subtracted from the mix.

In some embodiments, all clients in a virtual room who are nottransmitting audio receive the same virtual room mix, but all clients ina virtual room who are transmitting receive the unmixed streams of theother priority transmitters, and then, either on the client or the mixerserver 1520, the transmissions are mixed, so that the transmitters donot play their own transmissions.

Alternatively, in some embodiments, the virtual room can be split intotwo or more virtual room mixes, one for half the participants in avirtual room, half for the other. The mixer server 1520 sends the mix ofa first half of the virtual room that does not include the participant'sown audio and then unmixed streams of the priority transmitters from asecond half of the room that does include the participant. Furtherdivision of the virtual room into smaller fractions is possible in otherembodiments.

In a preferred embodiment, each participant is placed in a virtual room,with a limited number of participants, for example up to one hundredparticipants. Thus if there are fifty thousand total participants, thefifty thousand participants might be split into hundreds of virtualrooms, for example five hundred virtual rooms. The audio streams fromall members of each room would be collected by the mixer server 1520,and mixed into a single mix per room.

To create a mix for the entire virtual stadium 1500, the mixer server1520 combines all the room mixes in a single stadium mix. Alternatively.the mixer server 1520 could divide up the stadium into two or moremixes. Multiple mixes ensures that the mixer server 1520 is capable ofsending to the client at least one mix that does not include thatclient's own ‘voice.’ The clients own virtual room mix may then beprocessed separately via the mixer server 1520. Finally the virtualstadium mix and the virtual room mixes are combined on the client ormixer server 1520 to produce the complete participant mix associatedwith an individual participant.

It is to be understood that the techniques described herein may becombined. Virtual rooms or participants may be given positions invirtual space, so that nearby virtual rooms or participants can bespatialized, using standard spatialization techniques, but other rooms,too far away to benefit from spatialization, be treated as simpleambient mixes. In some embodiments, participants may be allowed to movewithin the virtual room to ‘talk to’ specific individuals within theroom, for instance by selecting any empty chair, or by navigating in avirtual space, using a standard seat selection interface, or otherstandard ways of positioning themselves in a virtual space. The clientdevice may receive a seat selection or user selection, and transmit aposition of the seat selection or a user identifier to the mixer server1520. The mixer server 1520, in response to receiving the seat selectionor the user identifier, may separately mix an audio stream associatedwith the user identifier, or may store the position associated with theclient device into memory.

In some embodiments, the client device may generate a graphical userinterface having one or more indicators and controls for the volume ofthe virtual stadium, virtual room mix or synthesized sounds. Thegraphical user interface may be one of many standard volume controlinterfaces, such as a slider or number, or it can be automaticallybalanced, lowering the virtual stadium sounds to ensure that priorityparticipants can be heard. In some embodiments, the graphical userinterface includes one or more indicators and controls for selecting avirtual room. The client device or mixer server 1520 may assign orsuggest to participants a preferred room, in some embodiments, based onwhich virtual room most of the friends or acquaintances of theparticipant are already in. In other embodiments, other participants mayinvite them to specific rooms, for instance the room the otherparticipant has already selected.

Like real events, virtual events in virtual stadiums can be recorded andreplayed. In a preferred embodiment, the audience participates in theevent as it is first broadcasted to that audience, in real time. Howeverthere also may be an audience for replayed events, or selected clipsfrom recorded events. The original or first audience response recording,if requested by the client, can be played in near synchronization withthe event replay.

A participant may not want to hear certain users. In some embodiments,if one participant indicates a particular user to mute or block, eitherpermanently or temporarily, via a graphical user interface of theassociated client device, for instance for the duration of the event,the client or mixer server 1530 may not play, or not transmit, orsubtract out the audio stream from the particular user's client devicefrom the mix.

An “usher” or event producer may wish to mute certain users, sections orrooms. In some embodiments, if an event administrator indicates aparticular user or room to mute or block, either permanently ortemporarily, via a graphical user interface of the associated clientdevice, for instance, for the duration of the event, the client or mixerserver 1530 may not play, or not transmit, or subtract out the audiostream of that user, section, or room.

In various embodiments, the mixer server 1520 expands the available‘room’ by creating multiple virtual (chat) rooms. As in other methodsfor audience participation, the mixer server 1520 can use severalvarious embodiments to assign users to virtual rooms, for instance,randomly, or by who invited them, or by which room contained the mostfriends, etc. In some embodiments, the mixer server 1520 collects thebest content, or highest rated content, from the various rooms andpromotes them to a final summary, or other rooms.

Significant delays between viewing an event, and responding to thatevent, are generally undesirable. For instance, if laughs for a jokecome in after that joke has ended and the next joke has started, theexperience is less enjoyable. Delays exist between each transmission inthe virtual stadium 1500. For example, a delay exists from an event inthe video source 1510 transmitted to, via the mixer server 1520, theterminals 1530. Another delay occurs between a user receiving thereceived video and audio signal and their audible reaction to thesignal. Finally delays may occur from recording the user audio stream,transmitting the audio stream to the mixer server 1520, mixing theappropriate audio streams together into single or multiple channels, andtransmitting the final mix or mixes back to the terminals 1530.Embodiments of the present disclosure minimize the delays from theinitial event to receiving participant reactions to the event.

In various embodiments, all client clocks are approximatelysynchronized, using one or more techniques described herein or any othersuitable pre-existing techniques. If the mixer server 1520 receives amedia signal from one or more terminals 1530 too late, or past apredetermined cutoff, for instance, two seconds, the late audio feedbackfrom the one or more terminals 1530 is ignored or otherwise not includedin the mix. If the media is not too late, then the terminal can transmitaudio back to the mixer server 1520. In certain embodiments, the mixerserver 1520 determines a particular time window associated with anevent, such as a touch down in a live stream of a football match, andmixes together the audio signals received within the time window otherthan audio signals received before and after the time window. The mixerserver 1520 may determine the time window by analyzing distribution of asuitable variable, such as pitch or volume, of all received audiofeedback, to determine a time when the event occurred, and center thetime window about that time.

If audio reactions are sent with the media stream, this guarantees thatthe audio will not be transmitted before the media. However if audioreactions are sent as a separate stream, then it is important that thereaction is not played before the corresponding media. In someembodiments, the time stamp of the reaction is compared against the timestamp of the currently playing media via the client device or mediaserver 1520. If the reaction is too early, the audio stream associatedwith the reaction is delayed until the proper timing in the media.

In normal media and messaging clients, it is important that listenerscan understand what is said. Because of that, techniques have beendeveloped to correct errors so that the audio is of sufficient qualityto be understood. Often buffering and retransmissions are employed toincrease signal quality. Some of these techniques can delay thetransmission.

In virtual events, it is not important that the crowd is understood, butit is important that the reactions be relatively synchronized to theevent. In some embodiments, faster, simpler protocols, such as UDP areemployed by the mixer server 1520. In some embodiments, audio packetsthat are packetized past a predetermined cutoff time, or audio packetswhich are lost, are ignored and not mixed. In such embodiments, if apacket is lost, a volume of adjoining packets can be smoothed so thatthe lost packet is not apparent.

In various embodiments, once the mixer server 1520 receives the audiostreams, the mixer server 1520 may also remove any audio that isreceived with too much delay. If there are sufficient streams that arenot too late, the mixer server 1520 may prioritize the most up-to-datestreams, so that the mixer server 1520 sends out a mix or mixes withrelatively up-to-date audio responses.

Other methods may also be useful for prioritization, besides timeliness.the mixer server 1520 may rate clients or users by importance, which themixer server 1520 uses to increase or decrease a volume associated withthe audio stream from each client, or, to remove or ignore the audiostream partially or entirely. The mixer server 1520 sets the importanceof each client by any suitable and relevant factors, such as ticketcost, fame of the user, etc.

In certain embodiments, clients may pay to be heard, with cash, points,or virtual gifts. These payments may enable entry, for some time period,into a virtual VIP room, that can be heard by the performers, or enablea certain amount of time to talk to the audience.

In some embodiments, if a broadcaster that is recording the live eventpauses live mode or temporarily stops streaming, the broadcast will befrozen. In such examples, in some embodiments designed for feedback, theaudience participation can continue over the paused video.

Furthermore, embodiments of the present disclosure incorporate layeringof audio for scaling and moderation. When creating audio for a virtualspace shared with many participants there may be additionalconsiderations for voice, besides whether users are in the same virtualroom, or nearby each other in virtual space.

In some implementations of large audio environments, the mixer server1520 uses a system of layering to separate and balance various audiosignals. In one implementation, the mixer server 1520 divides upparticipants into several audio layers, for instance: “my room's audio,”“the announcer's audio,” “the crowd's (the other rooms') audio,” and the“event audio” feed. Through either manual controls, like a mixing panel,or through an automatic process, the volumes of each layer is balanced.

In one implementation, the mixer server 1520 sets the audio levels ofeach layer, and each user, via the client device, may further adjust theaudio levels. Each layer may be encoded or transmitted separately sothat the volume of each layer can be adjusted separately.

Embodiments of the present disclosure further adjust for various typesof received audio input stream from the plurality of terminals 1530. Forexample, one or more of the audio input streams may be too loud, havebad feedback, or even contain offensive speech or noise. Before variousaudio channels are mixed together by the mixer server 1520, standardtechniques may be implemented to detect for feedback or adjust volume,and mute bad audio inputs. Standard techniques can be used to convertspeech into text, and check for offensive language. In oneimplementation, suspect audio signals are recorded, time stamped, markedto indicate which user or client transmitted the audio signal. Otheruseful metadata, like the text of the message, may be recorded as well.Suspect signals can be examined, in bulk or one at a time, manually orautomatically. For instance, the mixer server 1520 may employ a binarysearch technique. For instance, sixteen channels of audio can be mixedtogether and listened to. If a problem in one or more of the channels isdetected, the group of sixteen channels is split into two groups ofeight, or otherwise divided into separate smaller groups, for furtherchecking. If the mixer server 1520 identifies a problem with one of thegroups of eight, the identified group is again divided into two groups,e.g. groups of four signals. The dividing process is continued until theproblematic audio stream is identified and eliminated by the mixerserver 1520.

FIG. 16 illustrates a graphical user interface 1600 for implementingreaction to reaction chat overlays. In one implementation, picture inpicture responses are arranged like standard text chat, vertically, fromoldest on top, to newest on bottom, as shown in the chat overlay 315 inFIG. 6. To go forward in a discussion, a user may scroll, gesture orselect downwards. To go backwards in a discussion, the user may scroll,gesture or select upwards.

A linear arrangement of responses, however, may make it difficult toplace a response anywhere besides the location after the last response.A linear arrangement may be appropriate if the response is intended tocontinue the discussions created by the list of responses. However,sometimes users may want to respond, not to the whole thread ofmessages, but to a particular message. When users wish to respond to aparticular message, the standard technique is called a threadeddiscussion, or conversation threading.

Various embodiments of the present disclosure enable video discussionsor reactions to have responses, not just at an end of the list, but alsoresponses to responses or reactions to reactions. For example, thegraphical user interface 1600 may comprise an underlying media item1605, a chat overlay 1610, a reaction video 1615, and a plurality ofreactions 1620 to the reaction video 1615. Instead of displaying alinear list of responses, responses to the response are displayedhorizontally, such as reactions 1620.

In one or more embodiments, gestures, buttons or other suitable userinput are generated to distinguish between a user response ‘at the end’of a chat thread or a response to a specific item or reaction. Forexample, the graphical user interface 1600 may comprise a button 1625for facilitating associating a new response or reaction to a particularresponse or reaction in the chat overlay 1610. In response to receivingthe user input by a user actuating 1625, or otherwise touching/clickinga predetermined area adjacent to or over the response, or other suitableuser input, a new response or reaction is created and associated withthe corresponding response or reaction.

Depending on which button 1625 the user actuates, the response will bedisplayed in the location of the button 1625.

FIG. 17 illustrates an exemplary computer system 1700 that may be usedto implement some embodiments of the present invention. The computersystem 1700 of FIG. 17 may be implemented in the contexts of the likesof computing systems, networks, servers, or combinations thereof. Thecomputer system 1700 of FIG. 17 includes one or more processor units1710 and main memory 1720. Main memory 1720 stores, in part,instructions and data for execution by processor units 1710. Main memory1720 stores the executable code when in operation, in this example. Thecomputer system 1700 of FIG. 17 further includes a mass data storage1730, portable storage device 1740, output devices 1750, user inputdevices 1760, a graphics display system 1770, and peripheral devices1780.

The components shown in FIG. 17 are depicted as being connected via asingle bus 1790. The components may be connected through one or moredata transport means. Processor unit 1710 and main memory 1720 isconnected via a local microprocessor bus, and the mass data storage1730, peripheral device(s) 1780, portable storage device 1740, andgraphics display system 1770 are connected via one or more input/output(I/O) buses.

Mass data storage 1730, which can be implemented with a magnetic diskdrive, solid state drive, or an optical disk drive, is a non-volatilestorage device for storing data and instructions for use by processorunit 1710. Mass data storage 1730 stores the system software forimplementing embodiments of the present disclosure for purposes ofloading that software into main memory 1720.

Portable storage device 1740 operates in conjunction with a portablenon-volatile storage medium, such as a flash drive, floppy disk, compactdisk, digital video disc, or Universal Serial Bus (USB) storage device,to input and output data and code to and from the computer system 1700of FIG. 17. The system software for implementing embodiments of thepresent disclosure is stored on such a portable medium and input to thecomputer system 1700 via the portable storage device 1740.

User input devices 1760 can provide a portion of a user interface. Userinput devices 1760 may include one or more microphones, an alphanumerickeypad, such as a keyboard, for inputting alphanumeric and otherinformation, or a pointing device, such as a mouse, a trackball, stylus,or cursor direction keys. User input devices 1760 can also include atouchscreen. Additionally, the computer system 1700 as shown in FIG. 17includes output devices 1750. Suitable output devices 1750 includespeakers, printers, network interfaces, and monitors.

Graphics display system 1770 include a liquid crystal display (LCD) orother suitable display device. Graphics display system 1770 isconfigurable to receive textual and graphical information and processesthe information for output to the display device.

Peripheral devices 1780 may include any type of computer support deviceto add additional functionality to the computer system.

The components provided in the computer system 1700 of FIG. 17 are thosetypically found in computer systems that may be suitable for use withembodiments of the present disclosure and are intended to represent abroad category of such computer components that are well known in theart. Thus, the computer system 1700 of FIG. 17 can be a personalcomputer (PC), hand held computer system, telephone, mobile computersystem, workstation, tablet, phablet, mobile phone, server,minicomputer, mainframe computer, wearable, or any other computersystem. The computer may also include different bus configurations,networked platforms, multi-processor platforms, and the like. Variousoperating systems may be used including UNIX, LINUX, WINDOWS, MAC OS,PALM OS, QNX ANDROID, IOS, CHROME, TIZEN, and other suitable operatingsystems.

Some of the above-described functions may be composed of instructionsthat are stored on storage media (e.g., computer-readable medium). Theinstructions may be retrieved and executed by the processor. Someexamples of storage media are memory devices, tapes, disks, and thelike. The instructions are operational when executed by the processor todirect the processor to operate in accord with the technology. Thoseskilled in the art are familiar with instructions, processor(s), andstorage media.

In some embodiments, the computing system 1700 may be implemented as acloud-based computing environment, such as a virtual machine operatingwithin a computing cloud. In other embodiments, the computing system1700 may itself include a cloud-based computing environment, where thefunctionalities of the computing system 1700 are executed in adistributed fashion. Thus, the computing system 1700, when configured asa computing cloud, may include pluralities of computing devices invarious forms, as will be described in greater detail below.

In general, a cloud-based computing environment is a resource thattypically combines the computational power of a large grouping ofprocessors (such as within web servers) and/or that combines the storagecapacity of a large grouping of computer memories or storage devices.Systems that provide cloud-based resources may be utilized exclusivelyby their owners or such systems may be accessible to outside users whodeploy applications within the computing infrastructure to obtain thebenefit of large computational or storage resources.

The cloud is formed, for example, by a network of web servers thatcomprise a plurality of computing devices, such as the computing device1700, with each server (or at least a plurality thereof) providingprocessor and/or storage resources. These servers manage workloadsprovided by multiple users (e.g., cloud resource customers or otherusers). Typically, each user places workload demands upon the cloud thatvary in real-time, sometimes dramatically. The nature and extent ofthese variations typically depends on the type of business associatedwith the user.

It is noteworthy that any hardware platform suitable for performing theprocessing described herein is suitable for use with the technology. Theterms “computer-readable storage medium” and “computer-readable storagemedia” as used herein refer to any medium or media that participate inproviding instructions to a CPU for execution. Such media can take manyforms, including, but not limited to, non-volatile media, volatile mediaand transmission media. Non-volatile media include, for example, opticalor magnetic disks, such as a fixed disk. Volatile media include dynamicmemory, such as system RAM. Transmission media include coaxial cables,copper wire and fiber optics, among others, including the wires thatcomprise one embodiment of a bus. Transmission media can also take theform of acoustic or light waves, such as those generated during radiofrequency (RF) and infrared (IR) data communications. Common forms ofcomputer-readable media include, for example, a floppy disk, a flexibledisk, a hard disk, magnetic tape, any other magnetic medium, a CD-ROMdisk, digital video disk (DVD), any other optical medium, any otherphysical medium with patterns of marks or holes, a RAM, a PROM, anEPROM, an EEPROM, a FLASHEPROM, any other memory chip or data exchangeadapter, a carrier wave, or any other medium from which a computer canread.

Various forms of computer-readable media may be involved in carrying oneor more sequences of one or more instructions to a CPU for execution. Abus carries the data to system RAM, from which a CPU retrieves andexecutes the instructions. The instructions received by system RAM canoptionally be stored on a fixed disk either before or after execution bya CPU.

Computer program code for carrying out operations for aspects of thepresent technology may be written in any combination of one or moreprogramming languages, including an object oriented programming languagesuch as Java, Smalltalk, C++or the like and conventional proceduralprogramming languages, such as the “C” programming language or similarprogramming languages. The program code may execute entirely on theuser's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present technology has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Exemplaryembodiments were chosen and described in order to best explain theprinciples of the present technology and its practical application, andto enable others of ordinary skill in the art to understand theinvention for various embodiments with various modifications as aresuited to the particular use contemplated.

Aspects of the present technology are described above with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems) and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer program instructions. These computer program instructions maybe provided to a processor of a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a machine, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, create means for implementing the functions/acts specified inthe flowchart and/or block diagram block or blocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

The flowchart and block diagrams illustrate the architecture,functionality, and operation of possible implementations of systems,methods and computer program products according to various embodimentsof the present technology. In this regard, each block in the flowchartor block diagrams may represent a module, segment, or portion of code,which comprises one or more executable instructions for implementing thespecified logical function(s). It should also be noted that, in somealternative implementations, the functions noted in the block may occurout of the order noted in the figures. For example, two blocks shown insuccession may, in fact, be executed substantially concurrently, or theblocks may sometimes be executed in the reverse order, depending uponthe functionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts, or combinations of special purpose hardware andcomputer instructions.

While various embodiments have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. The descriptions are not intended to limit the scope of thetechnology to the particular forms set forth herein. Thus, the breadthand scope of a preferred embodiment should not be limited by any of theabove-described exemplary embodiments. It should be understood that theabove description is illustrative and not restrictive. To the contrary,the present descriptions are intended to cover such alternatives,modifications, and equivalents as may be included within the spirit andscope of the technology as defined by the appended claims and otherwiseappreciated by one of ordinary skill in the art. The scope of thetechnology should, therefore, be determined not with reference to theabove description, but instead should be determined with reference tothe appended claims along with their full scope of equivalents.

What is claimed is:
 1. A computer-implemented method, comprising:generating a graphical user interface on a touch surface display;receiving a plurality of media sequences, each media sequence having atleast one media item, the plurality of media sequences forming a virtualwall; displaying, via the graphical user interface, a first media itemof a first media sequence of the plurality of media sequences;automatically displaying each next media item of the first mediasequence after each currently displayed media item is complete; andautomatically displaying a first media item of a second media sequenceafter a last media item in the first media sequence is complete.
 2. Thecomputer-implemented method of claim 1, further comprising: detecting aswipe gesture on the touch surface display in a first direction along afirst axis; and displaying a next media item of a currently displayedmedia sequence.
 3. The computer-implemented method of claim 2, furthercomprising: detecting a swipe gesture on the touch surface display in asecond, opposite direction along the first axis; and displaying aprevious media item of a currently displayed media sequence.
 4. Thecomputer-implemented method of claim 3, further comprising: detecting aswipe gesture on the touch surface display in a third direction along asecond axis orthogonal to the first axis; and displaying a first mediaitem of a next media sequence.
 5. The computer-implemented method ofclaim 1, wherein the first media item of the second media sequence isdetermined based on the most recent media item of the second mediasequence displayed.
 6. The computer-implemented method of claim 1,further comprising: storing a position of the virtual wall into memory;determining adjacent media items to a currently displayed media item;wherein the determining adjacent media items includes determining a nextand previous media item of a currently displayed media sequence anddetermining a media item to be displayed for a next and previous mediasequence; and caching the adjacent media items for display.
 7. Thecomputer-implemented method of claim 1, wherein each media sequencerepresents at least one of a live stream and a camera feed.
 8. Thecomputer-implemented method of claim 7, wherein a last media item ofeach media sequence is associated with a live feed, and each other mediaitem of each media sequence is associated with previous recordings ofthe live stream.
 9. The computer-implemented method of claim 1, furthercomprising: generating a vertical and a horizontal navigation userinterface; and displaying, via the graphical user interface, thevertical and the horizontal navigation user interfaces.
 10. Thecomputer-implemented method of claim 9, wherein the vertical navigationuser interface includes a preview associated with each media sequence inthe virtual wall.
 11. The computer-implemented method of claim 9,wherein the horizontal navigation user interface includes a previewassociated with each media item in a currently displayed media sequence.12. The computer-implemented method of claim 1, further comprising:generating a chat overlay having at least one video reaction; anddisplaying, via the graphical user interface, the chat overlay over acurrently displayed media item.
 13. The computer-implemented method ofclaim 12, further comprising: determining a length of time based on thechat overlay; in response to the currently displayed media itemcompleting playback, determining a total time the currently displayedmedia item has been displayed; and based on determining the total timeis less than the length of time, looping the currently displayed mediaitem until the total time is greater than the length of time.
 14. Thecomputer-implemented method of claim 12, wherein displaying the chatoverlay comprises: playing the at least one video reaction over thecurrently displayed media item; and scrolling through each videoreaction of the at least one video reaction in sequence.
 15. Thecomputer-implemented method of claim 14, further comprising: in responseto playing a video reaction having audio, reducing a volume of thecurrently displayed media item.
 16. The computer-implemented method ofclaim 7, further comprising: recording, via a microphone, a first audiosignal associated with a user reaction to a live stream event;transmitting the first audio signal to a server; and receiving aplurality of audio signals from the server, the one or more audiosignals including an audio signal of the live event and a mixed audiosignal associated with a plurality of other user reactions.
 17. Thecomputer-implemented method of claim 1, wherein the first media sequenceis associated with a department in a shopping catalog, each media itemof the first media sequence is associated with a sub-department in theshopping catalog, and the second media sequence is associated with oneof the sub-departments.
 18. The computer-implemented method of claim 17,further comprising: detecting a swipe gesture on the touch surfacedisplay in a third direction along a second axis, the swipe gesturebeing performed over a currently displayed media item associated with afirst sub-department; and displaying a first media item of a mediasequence associated with the first sub-department.
 19. A system forimplementing a virtual wall, the system comprising: a processor; and amemory coupled to the processor, the memory storing instructions whichare executable by the processor to perform a method comprising:generating a graphical user interface on a touch surface display;receiving a plurality of media sequences, each media sequence having atleast one media item, the plurality of media sequences forming a virtualwall; displaying, via the graphical user interface, a first media itemof a first media sequence of the plurality of media sequences;automatically displaying each next media item of the first mediasequence after each currently displayed media item is complete; andautomatically displaying a first media item of a second media sequenceafter a last media item in the first media sequence is complete.
 20. Thesystem of claim 19, wherein each media sequence includes a varyingnumber of media items.